Skip to content

feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul - #707

Draft
ScriptedAlchemy wants to merge 3682 commits into
masterfrom
codex/tracedecay-total-redesign-plan-reopened
Draft

feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul#707
ScriptedAlchemy wants to merge 3682 commits into
masterfrom
codex/tracedecay-total-redesign-plan-reopened

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Replacement review for #421 after its accidental merge was reverted from master in 52a9aab. This commit has the exact tree of #421 head d4c67d4. Do not merge until explicitly authorized.

Issues this PR fixes

Refs #753 (source-identity + restart journey tests landed on branch at eae9328; keep draft until full product journey/CI)
Refs #792 (journey acceptance pending)
Refs #837 (journey acceptance pending)
Refs #838 (journey acceptance pending)
Refs #855 (journey acceptance pending)
Fixes #862
Fixes #861
Fixes #810
Fixes #845
Fixes #856
Fixes #853
Fixes #799
Fixes #836
Fixes #830

Partially addressed here, tracked to completion on their own issues: #852, #800, #863, #843, #842, #848.

🤖 Generated with Claude Code

#887 dead-surface deletion inventory

  • Deleted 2,479 lines across 93 files (31 insertions), including uncategorized test ports, obsolete schema/registry helpers, dead graph/LSP/session/storage APIs, and stale dead-code suppressions.
  • build_state compatibility wrapper was deleted; the live get_gc_meta suppression was removed. clean_c_comment / clean_c_doc_comment are retained because feat: TraceDecay V2 delivery (PR8–PR13) + daemon performance, memory, and reliability overhaul #707 now has production callers in C, C++, GLSL, Go, and Objective-C extractors.
  • Remaining suppressions are limited to schema-derive carriers, feature-gated cross-binary test helpers, and platform/lifetime ownership roles with explicit comments; no future/follow-up suppression remains.
  • Verification: cargo check --workspace --all-targets; cargo clippy --workspace --all-targets -- -D warnings; both pass on the integrated branch.

#888/#890/#891/#892 workflow cleanup

  • Agent-suite inventory: 18 files / 134 tests -> 17 files / 122 tests; duplicate Claude inventories and prose/tool-name gates were deleted in favor of canonical schema, installer, and live CLI authorities.
  • Bundled skills: 18 skills / 1,953 body lines -> 17 skills / 449 body lines; reference/script material 502 -> 89 lines. The mandatory using-tracedecay bootstrap was deleted, detailed mechanics moved to live tool help, and unique safety/domain rules remain narrowly routed.
  • Managed skills now author routing descriptions and positive/neighbor/negative validation explicitly, support no_skill_needed, and keep only actual host size/syntax limits.
  • Kiro/Claude persisted steering migrated from prose headings to ownership sentinels with update/uninstall compatibility for shipped markers.
  • Eval harness carries endpoint profiles hermetically (env+model only, mode 0400), strips commands from skill-free arms, supports repeated parallel pools, and emits routing/outcome/efficiency/over-trigger metrics. Current-profile validation: 20/20 runs successful, both control/treatment isolated, positive/negative scenarios parsed and graded, zero 429s.

@changeset-bot

changeset-bot Bot commented Aug 24, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ea098cb

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Post-reopen correction pushed in a23f86a: source-neutral background reconcile no longer advances the scheduler epoch and cancel in-flight text activation; mounted hook overflow uses the explicit invalidating path. Evidence: RED reproduced 0/1, GREEN 1/1 for ordinary_background_reconcile_does_not_supersede_in_flight_text_work; cargo check --lib --locked passed.

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

if [[ -n "${{ steps.target-runtime.outputs.runtime_library }}" ]]; then
companion_args+=(
--companion
"${{ steps.target-runtime.outputs.runtime_library }}=${{ steps.target-runtime.outputs.runtime_entry_name }}"

P1 Badge Install the bundled Linux runtime beside the binary

For Linux targets, this archive now contains libonnxruntime.so.1, and the verification step succeeds only while that companion remains beside the $ORIGIN-linked executable. However, install.sh lines 80-84 extracts the archive and copies only tracedecay into the install directory before deleting the temporary directory. On systems without a compatible system ONNX Runtime, binaries installed through the advertised script will therefore fail in the dynamic loader even for --version; install the companion library beside the executable or use a layout whose runtime search path matches the installed location.


{
"type": "json",
"path": "server.json",
"jsonpath": "$.version"

P1 Badge Advance the SDK version in release PRs

The new npm publication job derives its version from the packed sdks/typescript/package.json, but this release-please list updates only the root manifest, CLI manifest, and server.json; the SDK manifest and lockfile remain fixed at 0.1.0. Starting with the release after 0.1.0, the job will either fail its different-integrity check for @tracedecay/sdk@0.1.0 or no-op for identical bytes, so no SDK version corresponding to the new stable release is published.


const overview = useQuery({
queryKey: ['delivery', 'overview'],
queryFn: () => fetchEnvelope('/api/delivery/overview', DeliveryOverviewV1Schema),

P1 Badge Scope Delivery queries to the selected project

When the scope bar selects a project other than the active project, this query keeps the same cache key and continues fetching the unprefixed active-project route. The dashboard already provides scopeKey and scopedUrl, and the backend mounts /api/projects/{id}/delivery/overview; without using them here, the page labels the selected project while showing another project's Git, CI, and release data, and switching scopes does not trigger a refetch.


let digest = RegistrationDigest {
project_id: project_id.to_string(),
canonical_root: registration_root.to_path_buf(),
git_common_dir: git_common_dir.clone(),
tracked_branches,

P1 Badge Include Git remote identity in the registration digest

When origin changes via git remote set-url without touching branch metadata or store artifacts, this digest remains identical, so the cache returns at lines 130-131 before git_remote_url is recomputed at line 147. The registry consequently retains the old remote and its alias, causing remote-based cross-project resolution to reject the new identity or continue resolving the stale one until an unrelated artifact changes; include the normalized remote in the digest.

AGENTS.md reference: AGENTS.md:L159-L161


private sendCurrentDiagnostics(): void {
const uris = vscode.languages.getDiagnostics().map(([uri]) => uri);
this.sendChangedDiagnostics(uris);

P2 Badge Batch startup diagnostics instead of dropping the tail

At activation or language-client restart, getDiagnostics() can return more than 32 document URIs, but this passes the entire list to limitAdmittedNativeDiagnosticDocuments, which permanently slices it to the first 32. Documents after that boundary are never synchronized unless they later emit a diagnostic-change or open event, leaving TraceDecay's native diagnostic state incomplete indefinitely for larger workspaces; split the startup list into bounded batches instead.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ScriptedAlchemy added a commit that referenced this pull request Aug 24, 2026
fix(clippy): clear the workspace clippy failures on #707
ScriptedAlchemy added a commit that referenced this pull request Aug 25, 2026
Removing the re-export shims left their module doc comments orphaned in
`context` and `graph`. The rest are pre-existing lints that only fire under
`--all-features`, which compiles the `test-transport` support surface as
non-test code:

- the registered-test-server constructor gets the file's existing explicit
  `expect_used` allow, since `lib.rs` denies it only outside `cfg(test)`;
- `http_application_registry` is read by the `cfg(test)` capacity journey and
  only reads as dead under `test-transport` alone, so the allow is gated to
  `not(test)` rather than renamed to a discard;
- the hotpath cadence helpers keep `#[inline(always)]` with an explicit allow,
  preserving the author's intent instead of silently downgrading it.

The `daemon_suite` restart wait is now the original helper recovered from the
PR #707 head rather than a reconstruction: it additionally proves the runtime
readiness receipt authorizes the exact configuration the SDK selected.
ScriptedAlchemy added a commit that referenced this pull request Aug 26, 2026
ScriptedAlchemy added a commit that referenced this pull request Aug 26, 2026
The relocation fix that anchored this test above the package allocated a
`PathBuf` only to hand out a reference, which `-D warnings` rejects as
`unnecessary_to_owned` and which failed the Clippy gate on #707.

`validate_requested_workload` takes `&Path`, and `CARGO_MANIFEST_DIR` is a
literal, so `Path::new` yields a `&'static Path` with no allocation and no
temporary to borrow from.
ScriptedAlchemy added a commit that referenced this pull request Aug 26, 2026
…707-20260826-a1

perf(index): integrate post-#721 catch-up delta into #707
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

#733 landed Clippy-red. Helper branch (not merged, shared checkout untouched):

cursor/707-clippy-after-733 @ latest HEAD of this PR (186880e550)

Clears the #733 Clippy denials:

  • nest Unbound | Unavailable or-patterns
  • collapse nested hint/witness and session-drain ifs
  • box SessionGraphAttachmentStateV1::Attached so the enum is not 808B

Adopt when you want:

git fetch origin cursor/707-clippy-after-733
git merge --ff-only origin/cursor/707-clippy-after-733

I will not land this on #707.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Clippy helper is now on the Codex worktrees (still not landed on this PR head):

  • relocate /fast/projects/tracedecay/.codex-worktrees/root-package-relocation @ 806b8c74ef (cherry-pick of 3293045f53)
  • post-721 /tmp/tracedecay-integrate-post721-707-a1 @ 127ab57fae (ff to 186880e550 then same cherry-pick)

Same change as cursor/707-clippy-after-733. I will not push/merge this onto #707.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Isolated post-#721 verification (worktree /tmp/tracedecay-integrate-post721-707-a1, CARGO_TARGET_DIR=/tmp/td-post721-isolated-target):

Green (non-vacuous) on merge 3b1baaee82 / later 127ab57fae:

  • tracedecay-graph-db --lib 87
  • tracedecay-runtime-core --lib 677
  • tracedecay-usecases --lib 837 (+1 ignored)
  • tracedecay-code-index --lib 177 (+1 ignored)
  • sealed_publication_tests 3, daemon::scheduler 33, session_runtime 1, sealed_projection_deadline_tests 1
  • late-bind native_declared_topology_projection 12

Not green

  • --all-features cannot compile tracedecay-rusqlite-runtime (admission.rs:120): hotpath::mutex! vs std::sync::Mutex because graph-db enables hotpath/hotpath without tracedecay-rusqlite-runtime/hotpath.
  • daemon::tests::scheduler 11 passed / 11 failed. Failures are scheduler_config::*: ensure_worker_planprofile code-index worker plan was not installed during daemon bootstrap (plus one writer-gate timeout and one missing scheduler key). Production bootstrap.rs still calls install_profile_worker_plan; the test daemon path does not.

Clippy helper remains on this tree at 127ab57fae / relocate 806b8c74ef. Not pushed to this PR head.

ScriptedAlchemy added a commit that referenced this pull request Aug 27, 2026
Ports the missing pieces of commit b1e41b712 ("fix(build): land the
batching commit's missing consumers") onto #707: HookCompletedReadinessDistributions
moved into tracedecay-agent-hosts with pub(crate) fields, so
input_rows_processed, input_rows_dropped_at_cap, and events_considered
are no longer reachable outside the crate. Adds the three public
accessors.

source_event and input_rows_received (also added by the source commit)
are not ported: nothing on this branch reads them through an accessor
today - the root-crate benchmark still compares via
serde_json::to_value(&readiness_distributions)["field"], which does
not need them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cursor
cursor Bot changed the base branch from master to cursor/vendor-split-707-d6ed August 27, 2026 05:09
cursor Bot pushed a commit that referenced this pull request Aug 27, 2026
Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
@cursor
cursor Bot force-pushed the cursor/vendor-split-707-d6ed branch from 5d8f2a1 to c7fcf3f Compare August 27, 2026 05:52
ScriptedAlchemy added a commit that referenced this pull request Aug 27, 2026
…d6ed

chore(vendor): stacked vendor-only PR for #707 — generated vendor churn, land first
@ScriptedAlchemy
ScriptedAlchemy deleted the branch master August 27, 2026 06:24
@ScriptedAlchemy
ScriptedAlchemy changed the base branch from cursor/vendor-split-707-d6ed to master August 27, 2026 06:24
cursor Bot pushed a commit that referenced this pull request Aug 27, 2026
…rules

Co-authored-by: Zack Jackson <ScriptedAlchemy@users.noreply.github.com>
The in-process LocalStoreRuntimeResolverV1 keys project-session
authority by ProjectId and enrollment-root path bytes. A fixed
dashboard_graph_fixture id plus a non-canonical TempDir path
collided on macOS when authority composition re-registered the
canonical /private/var/folders root. Isolate HOME/profile/data
under TraceDecayStorageEnvGuard and mint the id from the
canonical temp root.
CI PR dogfood (run 34317967355) finished its journey then failed because
TERM-to-KILL was 5044ms against the harness 5s grace. The journal showed
retirement_reapers_join_start with deadline_remaining_ms=10138, so ~4.9s
of the 15s background budget was already spent joining invocation (the
code-index worker) and sibling owners before project-server drain.

Prepare-time cancel now stops code-index reconcile, retention ticks,
automation loops, and host-admission replay; those joins abort at the
2s task-abort deadline instead of waiting for the in-flight pass.
MCP servers refuse new background refresh at TERM. Each owner logs
elapsed_ms so the next hang names the lane.

Measured locally with the debug CLI: idle TERM-to-exit 364ms, small
indexed project 529ms (invocation 43ms, git store actors 1ms). CI
before: 5044ms forced kill. 150 shutdown/lifecycle lib tests passed;
clippy -D warnings clean. 5s harness grace unchanged.
The engine type exists only on unix because the daemon serves a Unix
socket. Importing it unconditionally broke Windows libtest with E0432.
Gate the import and the runtime_identity suite that requires the engine.
The scoped observation reset destroyed the stream but left the
external-source receipt journal. Re-admission of unchanged transcripts
reused the same logical-effect keys with new request digests (new
anchors, cursors, sanitization receipts), so every filler file
conflicted and the worker retried the batch on its 1.5s cadence.
That is why 24 rollouts took ~40s and 240 took ~6 min: linear in
conflicting files, not a journal TTL. Convergence happened once the
replaced-identity session (new observation ids) got through and the
sweep finished, not because receipts expired.

The reset already retires every other derivation of the destroyed
stream. Receipts attest those commits, so they reset with it under
the exclusive maintenance transaction. Re-admission is a rebuild,
not a supersession of a still-true prior command. Writer-ledger
rows named `external-source.*` retire with them; foreign keys stay.

Refs #1130
macOS tempfile mounts resolve through /var -> /private/var, so a lexical
StoreLayout locator and the opened database path named one file as two.
same_canonical_path already exists for that host spelling pair.
normalize_path_text canonicalized tempfile roots to /private/var while
production session keys keep the public /var spelling, so pinned-project
search missed the Hermes hit on macOS.
macOS /var firmlinks plus an append-only (write=false) open raced ENOENT
on concurrent jsonl create. Canonicalize the existing prefix and open
with write+append so the ambient cap-std walk sees one path.
macOS accept inherits O_NONBLOCK from the non-blocking GraphQL listener,
so the first stream.read() returned WouldBlock (os 35) instead of the
request.
Starved 3-vCPU macOS runners exceeded the 350ms initialize window, so
hang and partial-frame tests reported StartupFailed instead of the later
TransportFailed / InvalidResponse. Hang and quiet budgets stay 350ms/50ms.
FSEvents can mark dirty during watch install; shutdown joins before
debounce drains, so requiring is_clean() failed on macOS. Compare the
dirty snapshot around the post-shutdown commit and drop the 500ms sleep.
git worktree add --detach failed with ENOENT on .git/worktrees/*/locked
when a parent process leaked GIT_DIR. Clear those vars like production
git.rs and create the worktrees parent before add.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment